Skip to content

Msf::Module::UUID#generate_uuid: Replace Rex::Text with SecureRandom.uuid #20170

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

bcoles
Copy link
Contributor

@bcoles bcoles commented May 12, 2025

Module UUIDs were added in commit 9277f06 titled "Store a uuid for each module, track this in sessions" in 2011. Module UUIDs differ between Metasploit runs, as they are dynamically generated at runtime using Msf::Module::UUID#generate_uuid.

This method was authored in 2011 when Metasploit contained far fewer modules. Now this method is called approximately 27 thousand (!) times during startup. Due to the ever-increasing number of modules, this behaviour will cause the startup time to grow.

The UUIDs are generated with Rex::Text.rand_text_alphanumeric(8).downcase, which does not generate standard UUIDs.

self.uuid = Rex::Text.rand_text_alphanumeric(8).downcase

The generated UUID is 8 characters in length (64bit). Most of the key space is wasted, as only 36 values (a-z and 0-9) of 256 are used.

Generating collisions is unlikely, and the impact to startup time is minimal, but the computation is needlessly expensive and wasteful:

  • rand_text_alphanumeric returns mixed-case alphanumic characters, but we ultimately discard uppercase A-Z
  • rand_text_alphanumeric constructs an array, then calls Rex::Text#rand_base,[1] which:
    • needlessly deals with bad characters,[2] of which there are none
    • performs a bunch of expensive array operations (join, pack, uniq) and appends to an array in a loop[2]

All we really want is a unique value that is unlikely to cause collisions. If UUID collisions or speed were an issue we could pre-compute values (which has the added benefit of consistency between Metasploit runs).

Instead, this PR replaces Rex::Text.rand_text_alphanumeric with SecureRandom.uuid which is significantly faster (almost 10x 🚀):

#!/usr/bin/env/ruby
require 'benchmark'
require 'securerandom'
require 'rex/text'

n = 100_000

Benchmark.bm(20) do |bm|
  bm.report('Rex::Text.rand_text') do
    n.times { uuid = Rex::Text.rand_text_alphanumeric(8).downcase }
  end

  bm.report('SecureRandom.uuid') do
    n.times { uuid = SecureRandom.uuid }
  end
end

Benchmarked on a system with 2 cores and 4GB RAM:

  • 100,000 iterations:
                           user     system      total        real
Rex::Text.rand_text    2.484292   0.017122   2.501414 (  2.522912)
SecureRandom.uuid      0.300432   0.040074   0.340506 (  0.341532)
  • 27,000 iterations:
                           user     system      total        real
Rex::Text.rand_text    0.681830   0.003688   0.685518 (  0.692985)
SecureRandom.uuid      0.083827   0.011939   0.095766 (  0.095777)

This does not introduce an extra dependency as we already use SecureRandom in Framework:

# grep -rn securerandom lib/
lib/msf/base/sessions/encrypted_shell.rb:2:require 'securerandom'
lib/msf/base/sessions/mettle_config.rb:4:require 'securerandom'
lib/msf/core/modules/external/message.rb:4:require 'securerandom'
lib/msf/core/exploit/remote/tincd_exploit_client.rb:1:require 'securerandom'
lib/msf/core/db_manager/user.rb:2:require 'securerandom'
lib/msf/core/module/uuid.rb:2:require 'securerandom'
lib/msf/core/payload/windows/encrypted_payload_opts.rb:1:require 'securerandom'
lib/msf/core/web_services/json_rpc_app.rb:1:require 'securerandom'
lib/msf/core/web_services/metasploit_api_app.rb:1:require 'securerandom'
lib/metasploit/framework/spec/threads/logger.rb:5:require 'securerandom'
lib/metasploit/framework/obfuscation/crandomizer/utility.rb:2:require 'securerandom'
lib/rex/post/meterpreter/pivot.rb:4:require 'securerandom'
lib/rex/payloads/meterpreter/config.rb:4:require 'securerandom'

Note: I'm not sure what effect this change will have on Metasploit Pro, if any.
Note: Maybe there is a reason we only want UUIDs to be only 8 characters? If so, we could simply truncate the generated UUID. This would still be much faster than using Rex::Text.


[1] https://github.com/rapid7/rex-text/blob/0d30d394c4378dbaddf7b489f0d22c2db4024ec7/lib/rex/text/rand.rb#L110-L118
[2] https://github.com/rapid7/rex-text/blob/0d30d394c4378dbaddf7b489f0d22c2db4024ec7/lib/rex/text/rand.rb#L144-L151

@bcoles bcoles force-pushed the msf-module-uuid branch from a1f0ea1 to 80222c8 Compare May 15, 2025 14:13
@bcoles bcoles requested a review from sjanusz-r7 May 17, 2025 01:16
@adfoster-r7
Copy link
Contributor

The UUID that isn't a UUID has always confused me; I'm happy to give this a go 🤞

Note: I'm not sure what effect this change will have on Metasploit Pro, if any.

I'm hoping this is just treated as an opaque value so it shouldn't cause any issues - but I'll run the test suite behind the scenes and see what happens 👍

@adfoster-r7
Copy link
Contributor

Thanks for the PR!

Just spent a few more cycles on this - including accidentally testing on Ruby 2.7.2 and things were much slower for me:

bundle exec ruby ./speed.rb
                           user     system      total        real
Rex::Text.rand_text    1.655100   0.004406   1.659506 (  1.661438)
SecureRandom.uuid      0.909573   5.385728   6.295301 (  8.649502)

But thankfully Ruby 3.4.1 is faster these days 😄

$ bundle exec ruby ./speed.rb
./speed.rb:1: warning: benchmark was loaded from the standard library, but will no longer be part of the default gems starting from Ruby 3.5.0.
You can add benchmark to your Gemfile or gemspec to silence this warning.
                           user     system      total        real
Rex::Text.rand_text    2.300610   0.019374   2.319984 (  2.342445)
SecureRandom.uuid      0.225130   0.002251   0.227381 (  0.227645)

I think using a real UUID will cause issues with the current database constraints that are implemented for task objects:

t.string "module_uuid", limit: 8

We could likely remove/change this constraints etc - but I don't know if that'll cause any issues for external tools using this value with similar limits/assumptions applied etc.


Looking at a slightly different/alternative approaches - what are your thoughts on keeping the same generation logic for now; but using a faster way of generating the same values that Metasploit currently uses 🤔

# frozen_string_literal: true

require 'benchmark'
require 'securerandom'
require 'rex/text'

module FasterUuid
  UUID_CHARS = [*('a'..'z'), *('0' .. '9')].freeze
  private_constant :UUID_CHARS

  def self.fast_uuid
    UUID_CHARS.sample(8).join
  end
end

n = 100_000

Benchmark.bm(20) do |bm|
  bm.report('Rex::Text.rand_text') do
    n.times { uuid = Rex::Text.rand_text_alphanumeric(8).downcase }
  end

  bm.report('SecureRandom.uuid') do
    n.times { uuid = SecureRandom.uuid }
  end

  bm.report('FasterUuid.fast_uuid') do
    n.times { uuid = FasterUuid.fast_uuid }
  end
end

Ruby 3.4.1

                           user     system      total        real
Rex::Text.rand_text    2.217110   0.027752   2.244862 (  2.292584)
SecureRandom.uuid      0.217144   0.002267   0.219411 (  0.219619)
FasterUuid.fast_uuid   0.103164   0.000521   0.103685 (  0.103891)

Using gem 'benchmark-ips' - https://github.com/evanphx/benchmark-ips gives us a more granular view:

# frozen_string_literal: true

require 'securerandom'
require 'rex/text'
require 'benchmark/ips'

module FasterUuid
  UUID_CHARS = [*('a'..'z'), *('0' .. '9')].freeze
  private_constant :UUID_CHARS

  def self.fast_uuid
    UUID_CHARS.sample(8).join
  end
end

Benchmark.ips do |bm|
  bm.report('Rex::Text.rand_text') do
    Rex::Text.rand_text_alphanumeric(8).downcase
  end

  bm.report('FasterUuid::fast_uuid') do
    FasterUuid::fast_uuid
  end

  bm.report('SecureRandom.uuid') do
    SecureRandom.uuid
  end

  bm.compare!
end

Ruby 2.7.2

$ bundle exec ruby ./speed.rb
ruby 2.7.2p137 (2020-10-01 revision 5445e04352) [x86_64-darwin20]
Warming up --------------------------------------
 Rex::Text.rand_text     4.985k i/100ms
FasterUuid::fast_uuid
                        98.877k i/100ms
   SecureRandom.uuid   903.000 i/100ms
Calculating -------------------------------------
 Rex::Text.rand_text     48.631k (± 4.0%) i/s   (20.56 μs/i) -    244.265k in   5.031811s
FasterUuid::fast_uuid
                        793.478k (±12.5%) i/s    (1.26 μs/i) -      3.955M in   5.066235s
   SecureRandom.uuid      9.807k (±18.5%) i/s  (101.96 μs/i) -     46.956k in   5.048430s

Comparison:
FasterUuid::fast_uuid:   793478.4 i/s
 Rex::Text.rand_text:    48631.2 i/s - 16.32x  slower
   SecureRandom.uuid:     9807.3 i/s - 80.91x  slower

Ruby 3.4.1

$ bundle exec ruby ./speed.rb
ruby 3.4.1 (2024-12-25 revision 48d4efcb85) +PRISM [x86_64-darwin23]
Warming up --------------------------------------
 Rex::Text.rand_text     5.877k i/100ms
FasterUuid::fast_uuid
                       129.473k i/100ms
   SecureRandom.uuid    45.866k i/100ms
Calculating -------------------------------------
 Rex::Text.rand_text     49.750k (±16.9%) i/s   (20.10 μs/i) -    246.834k in   5.119872s
FasterUuid::fast_uuid
                          1.029M (±10.7%) i/s  (971.54 ns/i) -      5.179M in   5.105812s
   SecureRandom.uuid    560.059k (±13.7%) i/s    (1.79 μs/i) -      2.752M in   5.015069s

Comparison:
FasterUuid::fast_uuid:  1029295.9 i/s
   SecureRandom.uuid:   560058.7 i/s - 1.84x  slower
 Rex::Text.rand_text:    49750.3 i/s - 20.69x  slower

Or making the value entirely lazy might work too, and we just generate the value when it's accessed? 🤔

diff --git a/lib/msf/core/module.rb b/lib/msf/core/module.rb
index ff455d4716..aed1b01b17 100644
--- a/lib/msf/core/module.rb
+++ b/lib/msf/core/module.rb
@@ -113,7 +113,6 @@ module Msf
       @module_info_copy = info.dup
 
       self.module_info = info
-      generate_uuid
 
       set_defaults
 
diff --git a/lib/msf/core/module/uuid.rb b/lib/msf/core/module/uuid.rb
index 5b3299d3df..523164c0bf 100644
--- a/lib/msf/core/module/uuid.rb
+++ b/lib/msf/core/module/uuid.rb
@@ -5,9 +5,10 @@ module Msf::Module::UUID
   # Attributes
   #
 
-  # @!attribute [r] uuid
-  #   A unique identifier for this module instance
-  attr_reader :uuid
+  # @return [String] A unique identifier for this module instance
+  def uuid
+    @uuid ||= generate_uuid
+  end
 
   protected
 
@@ -18,12 +19,8 @@ module Msf::Module::UUID
   # @!attribute [w] uuid
   attr_writer :uuid
 
-
-  #
-  # Instance Methods
-  #
-
   def generate_uuid
-    self.uuid = Rex::Text.rand_text_alphanumeric(8).downcase
+    @uuid = Rex::Text.rand_text_alphanumeric(8).downcase
   end
 end

Potentially combining both approaches could work, i.e. faster 'uuid' generation, as well as making it lazy could work - but I'm not strongly opinionated; there might alternative options too - just let me know if you've any preferences on next steps 💯

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants